Linear Co-occurrence Rate Networks (L-CRNs) for Sequence Labeling
نویسندگان
چکیده
Sequence labeling has wide applications in natural language processing and speech processing. Popular sequence labeling models suffer from some known problems. Hidden Markov models (HMMs) are generative models and they cannot encode transition features; Conditional Markov models (CMMs) suffer from the label bias problem; And training of conditional random fields (CRFs) can be expensive. In this paper, we propose Linear Co-occurrence Rate Networks (L-CRNs) for sequence labeling which avoid the mentioned problems with existing models. The factors of L-CRNs can be locally normalized and trained separately, which leads to a simple and efficient training method. Experimental results on real-world natural language processing data sets show that L-CRNs reduce the training time by orders of magnitudes while achieve very competitive results to CRFs.
منابع مشابه
Comparison of local and global undirected graphical models
CRFs are discriminative undirected models which are globally normalized. Global normalization preserves CRFs from the label bias problem which most local models suffer from. Recently proposed co-occurrence rate networks (CRNs) are also discriminative undirected models. In contrast to CRFs, CRNs are locally normalized. It was established that CRNs are immune to the label bias problem even they a...
متن کاملLow-Rank Hidden State Embeddings for Viterbi Sequence Labeling
In textual information extraction and other sequence labeling tasks it is now common to use recurrent neural networks (such as LSTM) to form rich embedded representations of long-term input co-occurrence patterns. Representation of output co-occurrence patterns is typically limited to a hand-designed graphical model, such as a linear-chain CRF representing short-term Markov dependencies among s...
متن کاملPredicting Secondary Structures, Contact Numbers, and Residue-wise Contact Orders of Native Protein Structure from Amino Acid Sequence by Critical Random Networks
Prediction of one-dimensional protein structures such as secondary structures and contact numbers is useful for the three-dimensional structure prediction and important for the understanding of sequence-structure relationship. Here we present a new machine-learning method, critical random networks (CRNs), for predicting one-dimensional structures, and apply it, with position-specific scoring ma...
متن کاملPredicting secondary structures, contact numbers, and residue-wise contact orders of native protein structures from amino acid sequences using critical random networks
Predictions of one-dimensional protein structures such as secondary structures and contact numbers are useful for predicting three-dimensional structure and important for understanding the sequence-structure relationship. Here we present a new machine-learning method, critical random networks (CRNs), for predicting one-dimensional structures, and apply it, with position-specific scoring matrice...
متن کاملMulticasting in cognitive radio networks: Algorithms, techniques and protocols
Multicasting is a fundamental networking primitive utilized by numerous applications. This also holds true for cognitive radio networks (CRNs) which have been proposed as a solution to the problems that emanate from the static non-adaptive features of classical wireless networks. A prime application of CRNs is dynamic spectrum access (DSA), which improves the efficiency of spectrum allocation b...
متن کامل